Differentiable Path Tracer
The focus of my master's thesis, titled 'Exploring Differentiable Path Tracing for Image Relighting with Depth Information,' involved the creation of a Differentiable Path Tracer. This program helped me to have a deeper understanding of various concepts and conduct experiments related to image relighting.
What is differentiable path tracing
The core idea behind differentiable rendering is that rendering a scene is just a function that takes in the scene parameters and outputs an image. If this function can be differentiated, we can calculate how a small perturbation of the parameters would affect the resulting image. Through optimization, we can then adjust these parameters to minimize the difference between the rendered image and a target image. The implementation of and explanation of the differentiable path tracer is based on the following works [1] [2] [3] [4] [5] [6] [7] .
Images
Path tracer
Microfacet BRDF
The path tracer supports multiple material models. The microfacet model is a model that is physically based. The model implemented in this project was based on the Disney Burley 2012 model [8] , with the energy conservation adjustment based on [9] as suggested by [10] .
Bounding Volume Hierarchy
In order to get good performance, this path tracer uses a bounding volume hierarchy (BVH) constructed with the surface area heuristic better split the triangles. A simple example of this this data structure and ho it is generated top down is illustrated in the picture.
Multiple Importance Sampling
The Path tracer has importance sampling methods for the materials and Equirectangular HDR images. This allows it to apply multiple importance sampling (MIS), combining the BRDF samples with the samples from the distant environment light.
Scene optimization
The optimization scheme utilized by the differentiable path tracer was predominantly derived from the methodologies outlined in [5] , [6] , and [7] . as the capabilities of this differentiable path tracer listed below, are described in these referenced works.
Recursive Control Variates
Nicolet et al.[7] suggest the use of recursive control variates to decrease the variance of subsequent renders. In essence the control variate leverages information from a correlated render to reduce the error made by the Monte Carlo estimator. Consequently, The control variate is a better estimator compared to the original render.
Adam optimizer
Gradient Clipping
Gradient Masking
Progressive texture resolution
Multi-view optimization